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BACKGROUND OF THE INVENTION 

Tprhninal Fie.lrt of the Invention 

The present invention relates in general to the 
communications field, and in particular to an interaction of an 
electronic reading device with an address pattern. 
Description of Rela ted Art 

Numerous devices exist for accepting user input and 
controlling user interaction with desktop and portable computers, 
personal digital assistance (PDAs) , mobile phones, and other types 
of electronic devices. For example, a keyboard can be used to 
accept typed input and other types of commands, a mouse or a 
track-ball can be used to provide relative motion input as well 
as various types of point-and-click selections, a keypad can be 
used to provide input of numerical data and functional commands, 
navigational keys can be used for scrolling lists or otherwise 
repositioning a cursor, and various types of touchpads or 
touchscreens can be used to provide absolute positional coordinate 
inputs. Each type of mechanism for accepting input and for 
supporting user interaction has benefits and disadvantages in 
terms of size, convenience, flexibility, responsiveness, and easy 
of use. Generally, the selection of a particular type of input 
mechanism is dependent upon the function of the application and 
25 the degree and type of interaction required. 

With the ever expanding capabilities and availability of 
applications both on the Internet and the area of wireless 
technology, there continues to be a need to develop and provide 
new mechanisms for accepting input and interacting with users. 
In particular, some of the existing technologies suffer from 
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drawbacks or limitations, such as size and flexibility, that make 
them impractical and/or inconvenient to use in some situations* 
By expanding the range of mechanisms for supporting user 
interaction, application developers and end-users can have greater 
flexibility in the selection of input devices. Preferably, any 
such new mechanisms will provide increased flexibility and will 
maximize user convenience. In addition, the development of new 
mechanisms for interacting with users can expand the realm of 
potential applications . 

For example, while a keyboard typically provides a ^great 
deal of flexibility, particularly when it is used in connection 
with a mouse, a touchscreen, or other navigational device, its 
size makes it inconvenient in many cases, especially in the 
wireless context. 

Furthermore, electronic commerce is expected to be an area 
of growth in the next few years. Today there exists means to 
perform electronic commerce over the Internet using a personal 
computer, and the telecom industry is developing solutions to do 
the same via mobile terminals (phones) . Development of additional 
electronic commerce tools could provide even greater flexibility 
and convenience. 
SUMMARY OF THE INVENTION 

The present invention comprises a method, system, and 
apparatus for performing electronic commerce transactions. In 
accordance with the invention, a sensor in an electronic reading 
device detects portions of an address pattern. An electronic 
commerce client located in the electronic reading device or 
otherwise in communication with the electronic reading device 
converts the detected portions of the address pattern into 
electronic commerce transaction data. The electronic commerce 
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transaction data is then sent to an electronic commerce server for 
completing the electronic commerce transaction. 
BRIEF DESCRIPTION OF THE DRAWINGS 

For a more complete understanding of the present invention, 
reference is made to the following detailed description taken in 
conjunction with the accompanying drawings wherein: 

FIGURE 1 is a block diagram of a system in which an 
electronic pen can be used as an input device ; 

FIGURE 2 is a schematic diagram of a system for supporting 
use of the electronic pen described in connection with FIGURE 1; 

FIGURE 3 is an illustration of the protocol stacks that can 
be used in the case of local communications between an electronic 
pen and an electronic pen client; 

FIGURE 4 is an illustration of protocol stacks that can be 
used when an electronic pen and an electronic pen client 
communicate with one another via an Internet connection; 

FIGURE 5 is an illustration of a protocol stack for 
communications between an electronic pen client and each of the 
supporting entities when the electronic pen client is not located 
within a server on the Internet; 

FIGURE 6 is an illustration of protocol stacks that are used 
for communications between an electronic pen client and each of 
the supporting entities when the electronic pen client is located 
on the Internet; 

FIGURE 7 is a block diagram of the electronic pen logic that 
handles positions, strokes, actions, and grid descriptions; 

FIGURE 8 is a block diagram of a state machine for the 
electronic pen control block shown in FIGURE 7; 

FIGURE 9 is a block diagram of a state machine for an 
electronic pen client; 
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FIGURES 10A-10C are a message flow and signaling diagram 
illustrating the operation of the electronic pen system shown and 
discussed in connection with FIGURE 2; and 
DETAILED DESCRIPTION OF THE INVENTION 
5 The present invention relates to a system in which an 

electronic reading device, such as an electronic pen, an 
electronic mouse, or a hand scanner, works in cooperation with an 
address pattern (e.g., a specially formatted paper) to provide for 
a detection of a location of the electronic reading device over 

10 the address pattern. For instance, a pattern of dots can be 

defined such that, by examining a very small portion of the . 
pattern, a precise location in the overall pattern can be 
determined. In fact, it is possible to define a pattern that has 
the size of 73, 000,000, 000,000 A4 pages, which is equivalent to 

15 half the size of the entire United States. Portions of the pattern 

can be placed on sheets of paper or other objects. 

Then, using an electronic scanner pen that can detect the 
dots in the pattern, it is possible to detect the location of the 
pen with respect to the unique pattern. For example, when such 

20 a pen is used in connection with a specially formatted paper, the 

pen can detect its position (e.g., using a built in camera) by 
detecting a 3 mm by 3 mm portion of the pattern. By taking 
approximately 100 pictures per second, the pen is capable of 
determining its exact position to within 0 . 1 mm or less. This 

25 system can be used to provide user input, to facilitate user 

interaction, or to store handwritten notes or drawings. Moreover, 
by associating portions of the overall pattern with certain 
applications, such a system can be used to interact with wide 
variety of applications. 
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Referring now to FIGURE 1, there is illustrated an example 
of a system 2 in which an electronic pen 10 can be used as an 
input device. The electronic pen 10 includes an ink cartridge and 
is capable of writing in a typical fashion. The electronic pen 
10, however, includes some type of sensor (e.g., a built-in 
camera) that is used for detecting an address pattern on a 
specially formatted piece of paper 12. In particular, the paper 
12 is formatted with a small portion of a large address pattern 
such that when the electronic pen 10 is used to write on or 
otherwise make marks on the paper 12, the writings or markings can 
be electronically detected and stored. 

As an example, the paper 12 might constitute a form that can 
be used for sending an email. Thus, the paper 12 might include 
a space for writing in the email address of an intended recipient, 
a space for writing a subject of the email, and a space for 
writing the body of the email. As the electronic pen 10 is used 
to fill in each of the spaces, the position and movement of the 
electronic pen 10 on the paper 12 can be determined by repeatedly 
detecting the current x, y coordinates of the pen 10 (e.g., at 
rate of 100 frames per second) . The markings can then be 
converted into ASCII text using an appropriate handwriting 
recognition program. Once the user completes the form, the email 
can be sent, for example, by checking a send box at a 
predetermined location on the paper 12. 

Preferably, the coordinate information collected by the pen 
10 is sent by a short range radio transmitter in the electronic 
pen 10 to a nearby mobile station 14 using a short range radio 
interface 16 such as a local wireless radio link (e.g., a local 
wireless radio link, such as that is supported by Ericsson's 
Bluetooth™ wireless communications technology) . Alternatively, 
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instead of using a mobile station 14, the coordinate information 
could also be sent to, for instance, a desktop or portable 
computer, a personal digital assistant (PDA) , a television, or a 
Bluetooth terminal. Moreover, instead of using a local wireless 
radio link, other types of local wireless links, such as inductive 
coupling and infrared light; other types of radio links, such as 
Global System for Mobile Communication (GSM) ; or wired 
transmission media, such as a cable can also be used. The 
information can then be forwarded via an appropriate link, such 
as a cellular air interface 18, to a base station 20 or other 
network node. 

Referring now to FIGURE 2, there is illustrated a schematic 
diagram of a system 2 for supporting use of the electronic pen 10 
described in connection with FIGURE 1. Throughout the subsequent 
discussion, the system 2 is described primarily in connection with 
an electronic pen 10. It will be understood, however, that the 
invention and the underlying system 2 can instead use any type of 
electronic reading device, such as an electronic pen, an 
electronic mouse, or a hand scanner. As shown in FIGURE 2, the 
.system 2 includes six different entities, including the electronic 
pen 10, electronic pen client 22, a control node 24, a name server 
26, a base translator 28, and an application server 30. Although 
these various devices are described and depicted separately, it 
is also possible to combine two or more of the entities into the 
same device (e.g., the electronic pen 10 and electronic pen client 
22 can be contained in the same device) . 

The electronic pen 10 is responsible for detecting positions 
on the address pattern, producing actions, and sending information 
to the electronic pen client 22. In addition to being able to 
leave pen markings, some electronic pens can also have the ability 
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to produce other types of output, such as sound, vibration, or 
flashing lights. The electronic pen 10 includes a memory for 
storing a current grid, which comprises information relating to 
an area of the address pattern that is near the most recently 
5 detected position of the electronic pen 10. When the electronic 

pen 10 is loaded with the current grid, it knows what actions to 
take based on the positions that are read from the address 
pattern. When the electronic pen 10 is first turned on or when 
it moves to an area outside of the current grid, the electronic 

10 pen 10 must first request a new grid description before it can 

continue processing information. In such a situation, the 
electronic pen 10 requests a new grid description from the 
electronic pen client 22. 

The electronic pen client 22 can be located in a mobile 

15 station 14, in a PDA, in a desktop or portable computer, in the 

electronic pen 10 itself, in a server somewhere on the Internet, 
or in another device. The electronic pen client 22 serves as the 
center of communications in the overall system 2. In particular, 
the electronic pen client 22 receives new grid requests and action 

20 requests from the electronic pen 10 and responds to these requests 

by contacting an appropriate entity within the overall system 2 
to properly respond to the request from the electronic pen 10. 
Furthermore, when the electronic pen 10 is being used in 
connection with a particular application, the electronic pen 

25 client 22 can store the application and/or any corresponding data 

received from the electronic pen 10 to facilitate processing and 
use of the application. 

The name server 26 is used for translating a detected 
position on the address pattern into a Uniform Resource Location 

30 (URL) associated with that position. Different portions of the 
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address pattern are assigned to different applications. Neither 
the electronic pen 10 nor the electronic pen client 22, however, 
is aware of all of the different applications and the particular 
areas assigned to each application. Thus, when the electronic pen 
10 detects a new or unknown position, it forwards the position 
information to the electronic pen client 22, which in turn sends 
the information to the name server 26. The name server 26 then 
identifies an application associated with the received position 
and retrieves a URL where a description of the particular 
application can be found. The retrieved URL can then be used by 
the electronic pen client 22 to retrieve the application, 
description. 

As an alternative, the name server 26 can comprise a global 
name server that keeps track of a location, in the form of URLs 
to local name servers, where more information can be found about 
different addresses in the pattern. Similarly, each local name 
server can use other local name servers to obtain the necessary 
information, i.e., to convert a position into a URL where an 
application description can be found. At the lowest level, the 
local electronic pen client should know all the paper addresses 
that are within a specific application or applications. 

There are some services that should be available in the 
overall system 2 for which it is inconvenient or not feasible to 
support such services in the electronic pen 10 or the electronic 
pen client 22. In such a case, the base translator 28 can be used 
to support the services. For example, the base translator 28 
might contain handwriting recognition software for converting pen 
actions into text or for converting pen actions into a predefined 
set of symbols. When such services are needed, the electronic pen 
client 22 can send a request to the base translator 28 along with 
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the necessary data, and the base translator 28 can perform the 
requested service. 

Another entity in the system 2 is a control node 24. The 
control node 24 is used for responding to actions in a 
standardized way. For example, the control node 24 can be used 
to respond to certain generic functions, such as "cancel" or 
"submit" functions, in a consistent manner without regard to the 
particular application that is currently active. 

In addition, the control node 24 is used for creating 
streaming-like applications. For instance, some applications 
might require that the positions on the address pattern that are 
detected by the electronic pen 10 be immediately sent, upon 
detection, to the electronic pen client 22 for use by the 
application (i.e., the electronic pen 10 does not wait to transmit 
the position data until a complete stroke is detected or until a 
"send" field is touched) . One example is an application that is 
used to control an industrial robot in a warehouse. In such a 
case, the application description that is loaded onto the 
electronic pen server 22 can include instructions that all 
positions be streamed to a control node 24. As a result, the 
control node 24 can receive the positions in real time and can 
control the robot without waiting for the form (i.e., the current 
grid) to be completed. Thus, the control node 24 can perform a 
real-time translation from detected positions to a responsive 
action, such as moving an object (e.g., a robot, a valve, etc.) 
or controlling a process. 

The application server 30 is a regular web or wireless 
application protocol (WAP) server that supports an application 
associated with a particular area of the address pattern. The 
application server 30 stores an application description and 
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provides the application description to the electronic pen client 
22 upon request- In addition, the application server 30 receives 
input data from tha electronic pen 10 via the electronic pen 
client 22. For example, the application description might define 
5 a number of data entry areas on a form. Thus when data is entered 

on the form by the electronic pen 10, the data is received by the 
electronic pen client 22, converted into text using handwriting 
recognition software, and forwarded to the application server 30, 
which stores the data or otherwise processes the data in 

10 accordance with the. function of the application. 

Referring now to FIGURES 3 through 6 there are illustrated 
various examples of protocol stacks that can be used for 
communicating between the entities shown in FIGURE 2. Generally, 
however, such protocols apply however, only if the two 

15 communicating entities are implemented in different devices. If 

two or more entities are combined into one device, a proprietary 
protocol can be used to communicate between the entities. FIGURE 
3 illustrates the protocol stacks that can be used in the case of 
local communications (e.g., using Bluetooth) between the. 

20 electronic pen 10 and the electronic pen client 22. If, on the 

other hand, the electronic pen 10 and the electronic pen client 
22 communicate with one another via an Internet connection, the 
protocol stacks depicted in FIGURE 4 will be used. FIGURE 5 
illustrates a protocol stack for communicating between the 

25 electronic pen client and each of the supporting entities, such 

as the name server 26, the control node 24, the base translator 
28, and the application server 30, when the electronic pen client 
22 is not contained within a server on the Internet (e.g., such 
as when the electronic pen client 22 is located in a mobile phone 
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14) . Finally, FIGURE 6 depicts the protocol stacks that are used 
when the electronic pen client 22 is located on the Internet. 

There are a number of procedures that can be used by the 
various entities in the system 2 to allow the system to operate 
properly. When the electronic pen 10 detects a position on the 
address pattern that is not within its currently loaded grid or 
when the electronic pen 10 has no currently loaded grid, the 
electronic pen 10 initiates a new grid procedure. The new grid 
procedure involves sending a new grid request object to the 
electronic pen client 22. The new grid request object contains 
the newly detected position, a description of the actions that the 
electronic pen 10 can natively support, and a description of the 
output signals that the electronic pen 10 supports. The reply to 
a new grid request object is a grid description, which can be 
provided by the electronic pen client 22 from its own internal 
memory or from the information provided by an application server 
30. Generally, the electronic pen client 22 extracts the grid 
description from an application description received from the 
application server 30. The grid description should only contain 
action-field-types that the electronic pen 10 has indicated that 
it natively supports, which means that the electronic pen client 
22 in some cases should convert the extracted grid description 
into a format that the electronic pen 10 can understand. 

In some situations, it may be necessary for the electronic 
pen 10 to unload its current grid at the request of the electronic 
pen client 22. In such a case, the electronic pen client 22 sends 
an empty grid description to the electronic pen 10, thereby 
causing the electronic pen 10 to unload its current grid. This 
can occur, for example, when a particular application is complete 
or when a new grid description request received from the 
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electronic pen 10 cannot be fulfilled, such as when the position 
received from the electronic pen 10 is not registered in the name 
server 26. 

Another similar message is the empty grid description with 
a grid exception. When the electronic pen 10 requests a new grid 
description from the electronic pen client 22, the electronic pen 
client 22 uses the detected position specified in the request to 
ask the name server 26 for a URL where the application description 
can be found. If no URL is returned, the electronic pen client 
22 can send an empty grid description with a grid exception to the 
electronic pen 10. The grid exception comprises a rectangle or 
other shape indicating the area around the detected position where 
no registered applications can be found. Preferably, the 
indicated area is as large as possible so that the electronic pen 
10 and/or electronic pen client 22 know the extent of the 
surrounding area that is unassigned and do not have to repeatedly 
send requests to the name server 26. Thus, the empty grid 
description with a grid exception causes the electronic pen 10 to 
unload its current grid and also informs the electronic pen 10 of 
an area surrounding the detected position that can essentially be 
ignored because its is not associated with any application. 

The procedure that is used when the electronic pen 10 
detects a new position is a find application description location 
procedure. This procedure is used by the electronic pen client 
22 to translate a detected position received from the electronic 
pen 10 into a URL where a description of an application 
corresponding to that position can be found. The procedure 
involves sending a request from the electronic pen client 22 to 
the name server 26 containing identification of the detected 
position. The name server 26 responds by sending a reply to the 
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electronic pen client 22 containing a URL where an application 
description can be found or, if the detected position is not 
registered in the name server 26, containing an indication that 
no associated application is known to exist. 

Once the electronic pen client 22 knows the URL where an 
application description can be found, the electronic pen client 
22 can initiate a get application description procedure, which 
allows the electronic pen client 22 to retrieve the application 
description from the application server 30. In particular, the 
electronic pen client 22 sends an application description request 
containing a unique ID for the requesting electronic pen 10 and/or 
electronic pen client 22 to the application server 30 located at 
the URL address provided by the name server 26. In response, the 
application server 30 provides an application description object 
to the electronic pen client 22, which loads the application onto 
the electronic pen client 22. The application description object 
is similar to an HTML form with some additions and modifications. 

Furthermore, the application description object can be sent 
from the application server 30 to the electronic pen client 22 in 
response to a submitted form (i.e., a submission of one completed 
form might automatically result in a new form being loaded onto 
the electronic pen client 22) . A related procedure is the 
application submit procedure, which is used by the electronic pen 
client 22 when the user of the electronic pen 10 selects a 
"submit" field in a form. In response to the selection of the 
"submit" field, the electronic pen client 22 will submit the form 
content in accordance with instructions received in the 
application description. Typically, the electronic pen client 22 
will submit the form content, in the same way as a regular web 
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browser, to a URL specified in a form tag of the application 
description. 

When an action that can be handled by the electronic pen 10 
itself is generated, an action procedure is initiated by the 
electronic pen 10 to send an action request object to the 
electronic pen client 22. If the electronic pen client 22 cannot 
translate the action into a field value itself, the electronic pen 
client 22 further forwards the request to a base translator 28 for 
translating the action into a field value. In response to the 
action request object, an action reply object is sent from the 
electronic pen client 22 to the electronic pen 10. The action 
reply object contains output information that indicates to the 
electronic pen 10 which outputs signals to use. The output 
information, however, cannot be of type that the electronic pen 
10' has previously indicated that it does not support. In some 
instances, the action reply object might contain a new grid 
description. In such a case the electronic pen 10 will unload its 
current grid description and load the new grid description. 
Similarly, if the action reply object contains an empty grid, 
description, the electronic pen 10 will simply unload its current 
grid description. 

The action request object is also sometimes used to specify 
actions that should be processed by the control node 24 . In this 
instance, the electronic pen client 22 initiates a control 
procedure by forwarding the received action to the appropriate 
control node 24. As a result, the control node 24 sends an action 
reply object to the electronic pen client 22. 

The operation of the electronic pen 10 will now be discussed 
in greater detail. Each electronic pen 10 has a unique pen ID, 
which is sent to the application server 30 when an application 
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description is requested. The electronic pen ID allows the 
application to identify the particular user that is using the 
application and to distinguish between multiple concurrent users 
of the same application, such as when different electronic pens 
10 are being used in connection with separate sheets of paper that 
each contain the same portion of the address pattern. 

Referring now to FIGURE 7, there is illustrated a block 
diagram of the electronic pen logic that handles positions, 
strokes, actions, and grid descriptions for the electronic pen 10. 
The electronic pen 10 includes a control block 32 for controlling 
the operation of the electronic pen 10. A grid description block 
34 represents a memory location that stores a current grid 
description. At any given time, the electronic pen 10 can be in 
■ either of two modes. In a first mode, a grid description is 
15 loaded, while in a second mode, the grid description block 34 is 

not loaded with a current grid description. 

As the electronic pen 10 moves across an address pattern, 
the electronic pen 10 periodically (e.g., every 1/100 of a second) 
detects a position by detecting all of the dots within, for 
example, a 3mm by 3mm area. Each detected position is forwarded 
(as indicated at 36) to a position first in first out (FIFO) block 
38, which acts as a buffer for temporarily storing the detected 
positions. The clocking of the position FIFO block 38 is 
controlled by the control block 32 (as indicated at 40) . 
25 T he detected position is fed from the position FIFO block 38 

(as indicated at 42) to an in grid detector 44. The in grid 
detector 44 retrieves data from the grid description block 34 (as 
indicated at 46) and determines whether the received position is 
within the loaded grid description. If not, the in grid detector 
44 notifies the control block 32, which in turn initiates a 
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request for a new grid. When the detected position is within the 
current grid, the position is then sent (as indicated at 50) from 
the in grid detector 44 to a stroke engine 52. The stroke engine 
52 converts the received positions into strokes, which are then 
sent (as indicated at 54) to an action engine 56. A complete 
stroke is created when the electronic pen 10 is lifted from the 
paper or when it moves outside of the grid field where the stroke 
began. Finally, the action engine 56 converts the received stroke 
into an action that can be sent to the electronic pen client 22. 
By using grid action-f ield-types, the action engine knows which 
type of action to produce for a specific grid field. 

Referring now to FIGURE 8, there is illustrated a block 
diagram of a state machine for the control block 32 shown in 
FIGURE 7. In this figure, events are indicated in capital 
letters, while tasks associated with the event are depicted in 
brackets. The process starts at step 60 with a start up event 62, 
which causes the position FIFO block 38 to begin receiving 
detected positions. Initially, the electronic pen 10 is in a no 
grid loaded state 64, which means that the electronic pen 10 does 
not have a grid loaded in the grid description block 34. As a 
result, the control block 32 generates an outside grid indication 
66, thereby causing the electronic pen 10 to send the request for 
a new grid description to the electronic pen client 22 (i.e. , in 
accordance with the new grid procedure) and to stop the FIFO 
buffer 38. At this point, the electronic pen 10 enters a waiting 
for grid state 68 . 

Once the new grid has been received (as indicated at 70) , 
the control block 32 moves to a grid loaded state 72, at which 
time the new grid is loaded into the grid description block 34 and 
the position FIFO block 38 resumes operation. On the other hand, 
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if no grid is received (as indicated at 74), at least a portion 
of the positions stored in the FIFO buffer 38 are erased. Which 
part of the FIFO buffer to erase is determined by the grid 
exception area, if any, in the received empty grid description. 
Accordingly, all positions stored in the FIFO buffer 38 that are 
within the grid exception area should be erased. If no grid 
exception is received, the stroke associated with the position is 
erased. In addition, the FIFO block 38 resumes operation and the 
control block 32 moves into the no grid loaded state 64. 

When the control block 32 is in the grid loaded state 72, a 
current grid is loaded in the grid description block 34. While 
the control block 32 remains in this state 72, the position FIFO 
block 38 continues to receive detected positions and passes them 
on to the stroke engine 52 and action engine 56. Actions produced 
by the action engine 56 are sent (as indicated at 58) to the 
electronic pen client 22 (i.e., in accordance with the action 
procedure described above) . 

At some point, an outside grid indication 7 4 may be received 
by the control block 32 from the in grid detector 44 . The outside 
grid event 74 causes the FIFO block 38 to stop generating new 
positions. In addition, the electronic pen 10 enters a flushing 
stroke and action state 76 wherein the strokes that are currently 
in the stroke engine 52 and the actions that are currently in the 
action engine 56 are flushed to the electronic pen client 22. Once 
the stroke engine 52 and action engine 56 have been fully flushed 
(as indicated at 78), the electronic pen 10 sends a request for 
a new grid to the electronic pen client 22 and unloads the current 
grid. The control block 32 then moves back into the waiting for 
grid state 68. 
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As a general matter, the electronic pen 10 may be capable of 
supporting various different types of output, including audio, 
such as warning tones; visual, such as a flashing light; tactile, 
such as vibration; and/or ink. In some cases, it might be 
desirable to allow the user of the electronic pen 10 to turn off 
the ink of the pen 10, such as when the electronic pen is being 
used on a portion of the address pattern that is public or shared 
or when the user wants to be able to reuse the current sheet of 
paper. 

The electronic pen client 22 will now be described in 
greater detail. Generally, the electronic pen client 22 is 
analogous to a regular web browser. It is responsible for loading 
applications from application servers 30 and for handling input 
form the electronic pen 10. Preferably, the electronic pen client 
22 is located in a separate device from the electronic pen 10 
itself. This is because it is desirable to minimize the size and 
power supply requirements of the electronic pen 10, which will 
likely be adversely affected by the processing resources and 
memory necessary to support the functions of the electronic pen 
client 22. 

Referring now to FIGURE 9, there is illustrated a block 
diagram of a state machine for the electronic pen client 22. 
Initially, the electronic pen client 22 is in a no application 
loaded state 80. The electronic pen client 22 recognizes only one 
signal when in this state 80, namely a new grid request from the 
electronic pen 10. Such a request causes a load grid indication 
event 82. The electronic pen client 22 responds by sending a 
request to the name server 26 to translate a position contained 
within the new grid request into a URL where the application 
description can be found (i.e., in accordance with the find 
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application location procedure) . Next, the electronic pen client 
22 enters a waiting for application description URL state 84. If 
no URL for the application description can be found (as indicated 
at 86) , the electronic pen client 22 sends a new grid reply to the 
5 electronic pen 10, wherein the reply contains an empty grid 

description with a grid exception. As a result, the electronic 
pen client 22 returns to the no application loaded state 80. 

If a URL for the application description is received from 
the name server 26 (as indicated at 88) , the electronic pen client 
10 22 sends a request to the application server 30 to retrieve the 

application description (i.e., in accordance with the get 
application description procedure). Accordingly, the electronic 
pen client 22 enters a waiting for application description state 
90. 

15 if the electronic pen client 22 does not receive an 

application description from the application server 30 (as 
indicated at 92) , a new grid reply is sent by the electronic pen 
client 22 to the electronic pen 10 wherein the reply contains an 
empty grid. Thus, the electronic pen client 22 returns to the no 

20 application loaded state 80. If, however, the electronic pen 

client 22 does receive an application description from the 
application server 30 (as indicated at 94), the electronic pen 
client 22 sends a new grid reply to the electronic pen 10 
containing a new grid description, and the electronic pen client 

25 22 loads the application in its memory. In addition, the 

electronic pen client 22 moves into an application loaded state 
96. 

In the application loaded state 96, five types of actions 
can be received by the electronic pen client 22 from the 
30 electronic pen 10. First, a received action can include a request 
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10 



that the electronic pen client 22 cannot handle itself, in which 
case the electronic pen client 22 will send the action to the base 
translator 28 (as indicated at 98) . The electronic pen client 22 
then moves into a waiting for response from the base translator 
state 100. Once a base translator response 102 is received by the 
electronic pen client 22, the electronic pen client 22 updates a 
current form or other data associated with the currently loaded 
application and sends an action reply to the electronic pen 10 
with appropriate output information. 

Another type of action that the electronic pen client 22 can 
receive from the electronic pen 10 is a request that should be 
forwarded to a control node 24. In such a case, the action is 
sent to a control URL specified in the application description (as 
indicated at 104), and the electronic pen client 22 enters a 
15 waiting for response from the control state 106. Once a response 

is received from the control (as indicated at 108) , the electronic 
pen client 22 sends an action reply to the electronic pen 10 with 
appropriate output information. 

A third type of action is a submit form request, in response 
20 to which the electronic pen client 22 will submit the current form 

to the application server 30 that is identified by the URL in the 
application description (as indicated at 110) . The electronic pen 
client 22 then enters a waiting for response from the application 
server state 112. If the application server 30 responds by 
25 sending- an empty application description to the electronic pen 

client 22 (as indicated at 114), the current application is 
unloaded from the electronic pen client 22 and an action reply is 
sent to the electronic pen 10 with an empty grid. As a result, 
the electronic pen client 22 returns to the no application loaded 
state 80. On the other hand, if the application server 30 
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responds with a non-empty application description, the old 
application is unloaded from the electronic pen client 22, the new 
application description is parsed and loaded in the electronic pen 
client 22, an action reply is sent to the electronic pen 10 with 
a new grid description and with appropriate output information, 
and finally the electronic pen client 22 returns to the 
application loaded state 96. 

A fourth type of action that can be received by the 
electronic pen client 22 from the electronic pen 10 is a request 
to load a new grid. This action occurs, for example, when a 
position outside of the current grid is detected by the electronic 
pen 10. When a new grid request is received, the electronic pen 
client 22 sends a request to the name server 26 (as indicated at 
116) and the electronic pen client 22 returns to the waiting for 
application description URL state 84. 

Finally, a fifth type of action that can be received by the 
electronic pen client 22 is an action that the electronic pen 
client 22 can handle itself, in which case the electronic pen 
client 22 updates the current form and sends an action reply to 
the electronic pen 10 with appropriate output information (as 
indicated at 118) . The electronic pen client 22 then remains in 
the application loaded state 96. One type of action that the 
electronic pen client 22 might be able to handle itself is a local 
application. For example, the electronic pen client 22 might be 
capable of performing certain basic functions that are defined by 
a local application. Thus, when the electronic pen client 22 
receives a new grid request, the position associated with the new 
grid request can be analyzed to determine if it corresponds to a 
local application. If so, the electronic pen client 22 can load 
the application description from its local memory, send a new grid 
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description to the electronic pen 10 without having to communicate 
with the name server 26 or the application server 30. 

Another action that might be handled locally by the 
electronic pen client 22 relates to the selection of fields within 
a form. When the electronic pen client 22 receives an action, the 
field that corresponds to that action receives focus. When this 
occurs, the electronic pen client 22 might display the field's 
value on its display or output the value by audio. In addition, 
the electronic pen client 22 might allow the user to edit the 
value of the field by means other than the electronic pen 10. Yet 
another type of action that might be handled by the electronic pen 
client 22 itself are actions that relate to a clipboard function. 
When a "copy" field is selected, the value of the field that had 
focus at the time the copy field was selected is transferred to 
the clipboard. Similarly, when a "paste" field is selected, the 
value stored in the clipboard is transferred to the field that had 
focus at the time the paste field was selected. 

Referring now to FIGURES 10A through 10C, there is shown, by 
way of example, a message flow and signaling diagram illustrating 
the operation of the electronic pen system 2 depicted in and 
discussed in connection with FIGURE 2. Initially, the electronic 
pen 10 detects a first position on the address pattern at step 120 
(e.g. , at a location on a sheet of paper designated for composing 
and sending emails) . At this stage, it is assumed that the 
electronic pen 10 is in a no grid loaded state. Thus, in response 
to the detection of the first position, the electronic pen 10 
sends a new grid request 122, which contains the detected position 
information, to the electronic pen client 22. As a result, the 
electronic pen client 22 sends an application location request 124 
containing the detected position information to the name server 
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26, at step 126. The name server 26 translates the detected 
position into a URL where an application " description that 
corresponds to the detected position can be found (e.g., a URL 
address for a server containing an email application) , and returns 
an application location reply 128 containing the retrieved URL to 
the electronic pen client 22. 

The electronic pen client 22 then sends an application 
description request 130, which contains the unique pen ID for the 
electronic pen 10, to the application server 30. The application 
server 30 retrieves the application description at step 132 and 
sends an application description reply 134 containing the 
retrieved application description to the electronic pen client 22. 
The electronic pen client 22 then parses and stores the 
application description at step 136. This step further involves 
generating a current grid description from the application 
description and sending the grid description to the electronic pen 
10 in a new grid reply 138. The electronic pen 10 stores the 
received grid description at step 140 and resumes processing of 
the detected positions. Using the detected positions and the 
information in the grid description (e.g., so that the electronic 
pen 10 knows which fields of the email form are being filled in) , 
the electronic pen 10 generates strokes at step 142 and generates 
actions at step 144 using the stroke engine 52 and action engine 
56 shown in FIGURE 7. 

Each time an action is generated that cannot be handled by 
the electronic pen 10 itself, an action request 146 containing a 
description of the action is sent from the electronic pen 10 to 
the electronic pen client 22. At this point, the electronic pen 
client 22 should determine what type of action has been received 
so that it can respond to the action in an appropriate manner. 
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First, it is determined whether the action requires the attention 
of, or otherwise should be processed in accordance with, a local 
application at step 148. Very basic applications or frequently 
used applications (e.g., delete entered text) , for example, might 
be stored locally to avoid having to contact another entity. In 
such a case, the electronic pen client 22 retrieves the local 
application at step 150 and sends an action reply 152, which can 
contain a new grid description or other appropriate information. 

However, if it is determined at step 148 that the received 
action does not relate to a local application, the process 
continues at step 154 where it is determined whether the received 
action requires processing by an external translator (e.g., 
handwriting recognition) . If so, an action request 156 containing 
a description of the action is sent by the electronic pen client 
22 to the base translator 28. The base translator 28 processes 
the action at step 158 and sends an action reply 160 containing 
output information responsive to the received action (e.g. , text 
corresponding to written characters) to the electronic pen client 
22, which can forward the output information to the electronic pen 
10 in an action reply 162, if necessary. 

If it is determined at step 154 that the received action 
does not require processing by an external translator, it is next 
determined whether the action relates to a control application at 
step 164. If so, an action request 166 containing a description 
of the action is sent by the electronic pen client 22 to the 
control server 24. The control server 24 processes the received 
action at step 168 and, if a response is necessary, returns output 
information responsive to the received action in an action reply 
170, which is forwarded from the electronic pen client 22 to the 
electronic pen 10 in an action reply 172. 
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Assuming that it is determined at step 164 that the received 
action does not relate to a control function, it is next 
determined whether the action comprises a request to submit a form 
at step 174 (e.g., a selection of a "send" area on the email 
form) . If so, an action request 176 containing the data entered 
onto the form is sent by the electronic pen client 22 to the 
application server 30. The application server 30 processes the 
form at step 178 and sends an action reply 180 containing a new 
application description (or an empty application description) to 
the electronic pen client 22. The electronic pen client 22 parses 
and stores the new application description at step 182 and 
generates a new grid description from the newly received 
application description. The electronic pen client 22 then sends 
an action reply 18 4 containing the new grid description. Although 
not illustrated in the figure, the electronic pen 10 will 
typically respond to the receipt of a new grid description by 
unloading its current grid description and loading the new grid 
description into its memory. 

At some point, it is assumed that the electronic pen 10 
detects a position that is outside of the currently loaded grid 
at step 186. In response to such an event, the electronic pen 10 
sends a new grid request 188 containing the newly detected 
position data to the electronic pen client 22. In response, the 
electronic pen client 22 again generates an application location 
request 190 containing the detected position data and sends the 
request to the name server 26. The name server 26 determines 
whether a URL for an application description that corresponds to 
the newly detected position is available at step 192. 

If so, the name server 26 sends an application location 
reply 194 containing a retrieved URL to the electronic pen client 
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22, which in turn sends an application description request 196 
containing the unique pen ID for the electronic pen 10 to the 
application server 30 at the identified URL address, just as 
previously discussed in connection with messages 128 and 130. In 
this case, however, it is assumed that the application server 30 
determines that the requested application description is 
unavailable at step 198. As a result, the application server 30 
sends an application description reply to the electronic pen 
client 22 containing an empty application description. In 
response to the receipt of an empty application description, the 
electronic pen client 22 unloads the current application at step 
202 and sends a new grid reply 204 containing an empty grid 
description to the electronic pen 10. The electronic pen 10 
responds to the receipt of the empty grid description by unloading 
the current grid description at step 206. 

Another possibility is that the name server 26 determines at 
step 192 that a URL corresponding to the detected position is not 
available. In this situation, the name server 26 sends an 
application location reply 208 to the electronic pen client 22. 
The reply 208 may simply be empty to indicate that a URL is not 
available. Preferably, however, the reply 208 contains a grid 
exception defining the largest area possible around the detected 
position for which there is no corresponding URL. In response to 
the reply 208, the electronic pen client 22 sends a new grid reply 
210 containing an empty grid description with a grid exception. 
Upon receiving the reply 210, the electronic pen 10 unloads the 
current grid description at step 212. Furthermore, assuming that 
the electronic pen 10 receives and recognizes the grid exception 
information, the electronic pen 10 may subsequently be able to 
determine that certain detected positions on the address pattern 
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are not associated with any application without having to send a 
request to the name server 26 or the application server 30. 

The electronic reading device system makes it possible to 
send a digital copy what is written on a piece of paper or a 
whiteboard, for example, as an email, fax, or SMS. This is simply 
performed by writing the message on the specially formatted 
intelligent paper. This technology can be combined with e- 
commerce technology stemming from the mobile terminal development. 
The combination results in an electronic pen that is well-suited 
for e-commerce application. 

The electronic pen 10 can also be used to perform electronic 
commerce. It reguires, however, security algorithms, 

infrastructure solutions, and more to perform e-commerce in a 
secure and user-friendly manner with an electronic pen 10 (similar 
to e-commerce solutions with a mobile terminal) . 

The solution to performing electronic commerce with an 
electronic pen 10 is to introduce an electronic commerce client 
into the electronic pen 10 that is the same as is integrated into 
mobile terminals (this would most likely be the Mobile electronic 
Transactions (MeT) client) . The MeT client is a result of the MeT 
Initiative, which is further described in the "MeT Overview White 
Paper" and other information available at 
www . mobilet ransaction ■ org . Then, the electronic pen 10 can use the 
same infrastructure and standards as in mobile terminals, from a 
system point of view, to perform e-commerce. 

The user could then simply sign a note prepared according to 
this system 2 (i.e. , on a paper that includes an address pattern) 
to accept a purchase and mark with the electronic pen 10 on the 
note which credit card to use (perhaps by circling the credit card 
symbol) . The purchase would then be done electronically with the 
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electronic pen 10, paper, signature, and credit card information 
stored at an electronic commerce server on the Internet. Thus, 
the user will accept and perform the purchase by signing the note 
in an ordinary fashion as is done today but with no physical 
5 credit cards involved. In addition, the information about the 

transaction is performed wirelessly with the electronic pen 10 
connected to the Internet via either a mobile terminal or an 
access point. Furthermore, the electronic pen 10 could include 
intelligence such that the speed of writing, direction, and 
10 pressure of writing is known. This would enable a handwriting 

recognition algorithm stored in the server, electronic pen 10, or 
mobile phone 14 to safely verify a user's signature. 

Referring now to FIGURES 1 and 2, an illustrative embodiment 
will now be described. To perform an electronic commerce 
15 transaction, a user would use an electronic pen 10 to select a 

type of credit card and sign a note printed on an addressed paper 
12 associated with a particular vendor. The electronic pen 10 
would detect what information is written and send the detected 
information to a base translator 28 via an electronic pen client 
20 22 located in a mobile phone 14. A processor in the base 

translator 28 would then perform signature verification to 
authorize the transaction. The base translator 28 would also 
retrieve stored credit card information (e.g. , 'credit card number, 
expiration date, billing address, etc.) associated with the 
25 selected type of credit card for the user. Alternatively, the 

credit card information could be stored in the electronic pen 10, 
electronic pen client 22, or at another server on the Internet. 
Moreover, retrieval of the credit card information and/or 
signature verification might be performed in response to a request 
30 from an application server 30 associated with the addressed paper 
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12. The retrieved credit card information would then be forwarded 
to an application server 30 associated with the address pattern 
on the paper 12 to complete the transaction. Verification of the 
completed transaction could then be electronically sent to the 
vendor. 

Although various preferred embodiments of the method and 
apparatus of the present invention have been illustrated in the 
accompanying Drawings and described in the foregoing Detailed 
Description, it is understood that the invention is not limited 
to the embodiments disclosed, but is capable of numerous 
rearrangements, modifications, and substitutions without departing 
from the spirit of the invention as set forth and defined by the 
following claims. Furthermore, it shall be understood that the 
terms "comprises" and * comprising," when used in the foregoing 
Detailed Description and the following claims, specifies the 
presence of stated features, elements, steps, or components but 
does not preclude the presence or addition of one or more other 
features, elements, steps, components, or groups thereof. 
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WHAT IS CLAIMED IS: 

1. An electionic reading device, comprising: 

a sensor for detecting portions of an address pattern; 

and 

an electronic commerce client for communicating with a 
remote electronic commerce server to perform an electronic 
commerce transaction based on portions of the address pattern 
detected by the sensor. 

2. The electronic reading device of claim 1, further 
comprising a wireless local link transmitter for transmitting 
information relating to the electronic commerce transaction to the 
electronic commerce server. 

3. The electronic reading device of claim 2, wherein the 
information is transmitted to the .electronic commerce server via 
a separate electronic device. 

4. The electronic reading device of claim 1, wherein the 
electronic commerce client comprises an MeT client. 

5. The electronic reading device of claim 1, further 
comprising a processor for performing user verification. 

6. The electronic reading device of claim 5, wherein the 
processor performs user verification using a signature recognition 
functionality. 

7. The electronic reading device of claim 6, wherein the 
signature recognition functionality comprises correlating a 
signature written using the electronic reading device with at 
least one of a detected speed of writing, a detected writing 
pressure, and a detected direction of writing. 

8. The electronic reading device of claim 1, wherein the 
electronic commerce server comprises a credit card server. 
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9. A method of performing an electronic commerce 
transaction, comprising the steps of: 

detecting portions of an address pattern using an 
electronic reading device; 
5 converting the detected portions of the address pattern 

into electronic commerce transaction data; and 

sending the electronic commerce transaction data to an 
electronic commerce server for performing an electronic commerce 
transaction in accordance with the electronic commerce transaction 
10 data. 

10. The method of claim 9, wherein the electronic commerce 
server is compatible with MeT technology. 

11 . The method of claim 9, wherein the detected portions of 
the address pattern correspond to information written with the 

15 electronic reading device. 

12. The method of claim 11, wherein the information written 
with the electronic reading device comprises a user signature. 

13. The method of claim 12, further comprising the step of 
verifying the user signature. 

20 14. The method of claim 13, wherein the step of verifying 

includes at least one of determining a speed of writing, 
determining a writing pressure, and determining a direction of 
writing. 

15. The method of claim 9, wherein the step of sending 
25 comprises transmitting the electronic commerce transaction data 

via a wireless interface. 

16. The method of claim 9, further comprising the step of 
notifying a vendor of an outcome of the electronic commerce 
transaction. 
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